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I3XEA6E SENSOR COLOR CORRECTION 



FIELD OF THE INVENTION 

5 

This invention relates to correcting colors 
generated by image sensors. This invention is 
particularly useful for, but not limited to, 
correcting colors generated by solid-state image 
10 sensors such as Complementary Metal Oxide 

Semiconductor (CMOS) image sensors that are 
particularly prone to generating noise. 



15 BACKGROUND ART 

Color solid-state image sensors typically have a 
spectral sensitivity that differs from colors that 
are peceived by the human eye. Consequently, colors 

20 of an image perceived by a human eye tend to differ 

from colors generated by solid state image sensors. 
To improve the color reproduction accuracy, digital 
cameras with image sensors having Red, Green and Blue 
(R, G, B) output channels usually perform color 

25 correction with a 3x3 color correction matrix. 

Ideally, the matrix has matrix coefficients that are 
optimized to improve color fidelity. A method of 
adjusting matrix coefficients to improve color 
fidelity is described in published patent 

3 0 specification WO 0 0/10331. The method described may 

work well with Charged Coupled Device (CCD) sensors 
that have very low level of noise. However, CMOS 
image sensors inherently suffer from thermal noise 
and fixed pattern noise. Accordingly, this method of 

35 adjusting matrix coefficients is not ideally suited 
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to CMOS image sensors as the coefficients tend to 
amplify noise thereby significantly affecting image 
quality. 

5 The problem of noise amplification by color 

correction is reported in the publication 
^'Performance Analysis of a Color CMOS Photogate Image 
Sensor', IEEE Transactions on Electronic Devices Vol. 
47, NIJ January 2000. Noise performance is suggested 
10 to be resolved by improving the sensor quantum 

efficiency rather than by color correction methods. 

In the publication "Design of real-time Image 
Enhancement Preprocessor for CMOS Image Sensor'', IEEE 

15 Transactions on Consuicier Electronics, Vol.46 Nl , 

2000. Color correction hardware complexity is 

assessed rather than noise amplification problems. 
Complexity is an important issue especially if the 
color correction is to be implemented directly on the 

20 CMOS sensor chip. Nevertheless, the problem of noise 

affecting image quality is not addressed in this 
publication . 



Another approach to reduce color correction 
2 5 complexity is proposed in ''Color Signal Processing 

Technique for Single-Chip CCD Cameras That Employ 
CPUs with SIMD Instruction Sets", IEEE Transactions 
on Consumer Electronics, Vol.46 N2 , 2000. The color 
correction technique described operates directly on 
30 Color Filter Array samples. The technique is 

implemented as a vector operation and does not 
address noise amplification. 



35 



3 



CR10 83AC 



SUMMARY OF THE INVENTION 



According to one aspect of the invention there 
is provided a color correction circuit having at 
5 least three color input channel processing circuits, 

each of said color input channel processing circuits 
comprising : 

an adder with a corrected color channel 
output and an uncorrected color channel input; 
10 a noise reduction filter having a filter 

input coupled to said uncorrected color channel 
input ; 

an input channel multiplier having an input 
coupled to an output of said noise reduction 
15 filter, an output of said input channel 

multiplier being coupled to an input of said 
adder; and 

at least two further multipliers with 
inputs respectively coupled to outputs of other 
20 noise reduction filters foorming part of the 

other color input channel processing circuits, 
said two further multipliers having outputs 
coupled to inputs of said adder. 

25 Suitably, said multipliers for a color channel 

may have coefficients that when summed together are 
less than 0.2. Preferably, said coefficients for a 
color channel when summed together may be 
substantially zero. 



30 



Suitably, all said coefficients may be less than 
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Preferably, said noise reduction filter may be a 
Low Pass Filter. 
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According to another aspect of the invention 
there is provided a camera having at least three 
color input channel processing circuits, each of said 
5 color input channel processing circuits comprising: 

an adder with a corrected color channel 
output and an uncorrected color channel input; 

a noise reduction filter having a filter 
input coupled to said uncorrected color channel 
10 input; 

an input channel multiplier having an input 
coupled to an output of said noise reduction 
filter, an output of said input channel 
multiplier being coupled to an input of said 
15 adder; and 

at least two further multipliers with 
inputs respectively coupled to outputs of other 
noise reduction filters forming part of the 
other color input channel processing circuits, 
20 said two further multipliers having outputs 

coupled to inputs of said adder. 

Suitably, said camera may also comprise a color 
interpolation module coupled to said color input 
25 channel processing circuits. There may also be a 

gamma correction module coupled to said color 
interpolation module. 

According to another aspect of the invention 
3 0 there is provided a method of correcting a digital 

color sampled signal comprising at least three color 
channels, the channels being a Red Channel, Green 
Channel and Blue channel, the method comprising the 
steps of: 
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filtering uncorrected color sampled signals 
on each of said color channels to provide 
filtered channel sampled signals; 

multiplying said filtered channel sampled 
signals with selected coefficients to provide 
noise reduced signal samples of said uncorrected 
color sampled signals; and 

adding each of said uncorrected color 
sampled signals to selected said noise reduced 
signal samples to provide a corrected color 
sampled signal. 



Preferably/ the coefficients for each said 
channel when summed together may be suitably less 
15 than 0.2. Preferably, said coefficients for each 

said channel when summed together may be 
substantially zero. 



20 



Suitably, the method may be further 
characterised by performing the following expression: 
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Wherein, / and are respective color 

25 corrected samples of the uncorrected color sampled 

signals R,G,B; R,G,B are respective filtered 
channel sampled signals of the uncorrected color 
sampled signals R,G,B; and C^^ tO C33 are the 
coefficients with values less then 1. 
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Preferably, the coefficients C^^^^ C^^ and C 33 may 
be positive. 



5 BRIEF DESCRIPTION OF THE DRAWINGS 

In order that the invention may be readily 
understood and put into practical effect, reference 
will now be made to a preferred embodiment as 
10 illustrated with reference to the accompanying 

drawings in which: 

Fig. 1 is a block diagram of prior art 
color correction hardware; and 

15 

Fig. 2 is a block diagram of a camera in 
accordance with the present invention; and 

Fig. 3 is a block diagram of the image 
20 correction module in accordance with the present 

invention. 



DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT OF 
25 THE INVENTION 

With reference to Fig. 1 there is illustrated a 
block diagram of prior art color correction hardware 
as described in patent specification WO 00/10331. The 
3 0 function of the color correction hardware can be 

described by the following expression: 
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Where, R,G,B are uncorrected color samples 
generated at respective Red, Green and Blue output 
channels of a color image sensor; Rq. <^c ^c 
color corrected samples of the R,G,B digital samples; 
and a,, to a^^ are matrix coefficients that are 
optimized to improve color fidelity. The matrix 
coefficients are selected for a specific sensor's 
spectral sensitivity. The main diagonal (a^^, a^j, a^^) 
of the matrix of expression -(1) usually comprises 
coefficients that are greater than 1.0, whereas other 
coefficients in a row have values less that 1.0 and 
one or both other coefficients in a row may have 
negative values that are between -1.0 and 0.0. 
Hence, as will be apparent to a person skilled in the 
art, in expression -(1), color correction for a 
selected digital sample of a color channel (R,G,B) is 
typically performed by subtracting weighted values 
(intensity or power) of digital samples on the other 
color channels from a weighted value of the selected 
digital sample. To further illustrate this color 
correction approach, expression -(1) is expanded as 
follows : 

Rc = a^yR + a^^G + a^^,^ 

Gc = a2iR + a^fi + a^r^B — (2) 

Be = a..R + <332G + a^^B 
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An example of color correction coefficients for 
an image sensor (IS) is: Rc = + 1 . 2R - 0 . 5G - 0.25B 
for channel R; Gc= -0.15R + 1 . 4G - 0.2 5B for channel 
G; and Be = +0 . OIR - 0,3 5G + 1 . 7B for channel B. 
5 The coefficients are typically different for other 

image sensors having different spectral sensitivities 
to that of the image sensor IS. Unfortunately, this 
approach to color fidelity improvement is not ideally 
suited to CMOS image sensors as the matrix 
10 coefficients tend to amplify noise thereby 

significantly affecting image quality. 

Referring to Fig. 2 there is illustrated a block 
diagram of a digital camera 1. Camera 1 includes an 

15 image sensor 2, typically a CMOS image sensor, having 

three color channel outputs, these being Red, Green 
and Blue color channel outputs that are coupled to a 
color correction circuit 3 . The Red, Green and Blue 
color channel outputs provide uncorrected color 

20 sampled signals (R, G, B) to inputs of the color 

correction circuit 3 . The camera 1 also has a color 
interpolation module 4 and a gamma correction module 
5. The 3 and modules 4,5 perform image corrections to 
provide a resulting image of a desired quality 

25 comprising processed color sampled signals (R^, G^, 

Bp) on output channels of the gamma correction module 
5. The output channels of the gamma correction 
module 5 are tri-state and are coupled to by a data 
bus to a memory unit 6. Hence, in operation the 

30 processed color sampled signals (R^, G^, B^) are 

stored in the memory unit 6. 



There is also a control module 7 coupled to a 
display unit 8 in the form of a Liquid Crystal 
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Display for displaying displaying an image comprising 

the processed color sampled signals (R^, G^, B^) 

stored in the memory unit 6. The color interpolation 
module 4 and gamma correction module 5 are well known 
5 in the art and are therefore not described in detail. 

The color interpolation module 4 has three color 
input channels coupled to respective output channels 
of the color correction module 3. The gamma 

correction module 5 also has three input channels 

10 coupled to respective output channels of the color 

interpolation module 4. The output channels of the 
color correction module 3 provide corrected color 
sampled signals (R^, G^, B^) and the output channels 
of the color interpolation module 4 provide 

15 interpolated color sampled signals (R^, G^, B^) . 

The camera 1 also includes an user input module 
9 coupled to provide command signals to the control 
module 7. There is also an output port 10 coupled by 
a common bus 2 0 to the control module 7 and the 
memory unit 6 is coupled to the output port 10 by a 
suitable bus. Further, the sensor 2, color 

correction circuit 3, color interpolation module 4, 
gamma correction module 5 and memory unit 6 are 
coupled by the common bus 20. 

In use, images stored in the memory unit the 
that comprise processed color sampled signals (Rp/ 

Gp/ Bp) are displayed upon the display unit 8 in 
3 0 response to suitable commands provided from the user 

input module 9 . Such sampled signals are obtained by 
the control module 7 controlling the Sensor 2, color 
correction circuit 3, modules 4,5 and memory unit 6 



20 
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and display unit 8, Further, if user wished to 
transfer images stored on the memory unit 6, then by 
suitable commands effected by the user input module 
9 , images can be loaded onto a removable memory 
5 inserted into the output port 10. 

Referring to Fig. 3 there is illustrated a block 
diagram of the color correction circuit 3. The color 
correction circuit 3 has three color input channel 
10 processing circuits these being Red, Green and Blue 

color input channel processing circuits 31, 32 and 33 
that are coupled channel outputs of the image sensor 
2 to receive respective uncorrected color sample 
signals R,G,B. 

15 

The Red input channel processing circuit 31 
comprises an adder 34 with an output, that is the 
corrected color channel output providing the 

corrected color signal sample Rq, and an uncorrected 

2 0 color Red channel input directly coupled to the Red 

color channel output of the image sensor 2 . The Red 
Input channel processing circuit 31 also has a noise 
reduction filter which is a Low Pass Filter 3 5 with 
an input coupled to the uncorrected Red color channel 

2 5 input. There is also an input channel multiplier 3 6 

having an input coupled to an output of the Low Pass 
Filter 3 5 and an output of the input channel 
multiplier 36 is coupled to an input of the adder 34. 
The Input channel processing circuit 31 also includes 

3 0 two further multipliers 37, 38 with inputs 

respectively coupled to outputs of other noise 
reduction filters in the form of Low Pass Filters 39, 
40 forming part of the Green and Blue color input 
channel processing circuits 32,33. The two further 
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multipliers 37,38 have outputs coupled to inputs of 
adder 34, therefore adder 34 is a four bit adder with 
four inputs and a tri-state output. The common bus 
2 0 is also coupled to each of the color input channel 
5 processing circuits 31, 32,3 3 thereby allowing for 

different coefficients to be loaded into the 
multipliers 36 , 37 , 38 , 42 , 43 , 44 , 46 , 47 , 48 . 

The Green input channel processing circuit 32 
10 comprises an adder 41 with a corrected color channel 

output, that is the corrected color channel output 

providing the corrected color signal sample Gc,/ and 

an uncorrected color Green channel input G directly 
coupled to the Green color channel output of the 
15 image sensor 2. The Input channel processing 

circuit 32 also includes the noise reduction Low Pass 
Filter 39 with an input coupled to the uncorrected 
Green color channel input. There is also an input 
channel multiplier 42 having an input coupled to an 

2 0 output of the Low Pass Filter 3 9 and an output of the 

input channel multiplier 42 is coupled to an input of 
the adder 41. The Green Input channel processing 
circuit 32 also includes two further multipliers 43, 
44 with inputs respectively coupled to outputs of the 
25 other noise reduction Low Pass Filters 35, 40 forming 

part of the Red and Blue color input channel 
processing circuits 31,33. The two further 

multipliers 43,44 have outputs coupled to inputs of 
adder 41, therefore adder 41 is a four bit adder with 

3 0 four inputs and a tri-state output. 

The Blue input channel processing circuit 33 
comprises an adder 45 with a corrected color channel 
output providing the corrected color signal sample 
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Be,/ and an uncorrected color Blue channel input 

directly coupled to the Blue color channel output of 
the image sensor 2, The Input channel processing 
circuit 3 3 also includes the noise reduction Low Pass 
Filter 40 with an input coupled to the uncorrected 
Blue color channel input. There is also an input 
channel multiplier 46 having an input coupled to an 
output of the Low Pass Filter 40 and an output of the 
input channel multiplier 46 is coupled to an input of 
the adder 45. The Blue Input channel processing 
circuit 33 also includes two further multipliers 47, 
48 with inputs respectively coupled to outputs of the 
other noise reduction Low Pass Filters 35, 39 forming 
part of the Red and Green color input channel 
processing circuits 31,32. The two further 

multipliers 47,48 have outputs coupled to inputs of 
adder 45, therefore adder 45 is a four bit adder with 
four inputs and a tri-state output. 



20 The function of the color correction circuitry 3 

can be described by the following expression: 
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(3) 



to are identical to coefficients that 



Where a 

would be selected for an image sensor when using 
expression (1) ; and R,G,B are filtered samples of the 
uncorrected color samples R,G,B. 
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If the image sensor is identical to the image 
sensor as referred to above in relation to expression 
(1) and (2) , then coefficients on the main diagonal 
of the matrix are modified so each coefficient in the 
5 main diagonal is typically less than 1. Therefore, 

expression (3) in a more general form can be 
expressed as follows: 







1 


0 


0 


R 




^11 




^13 


R 






0 


1 


0 


G 


+ 






^23 


G 






0 


0 


1 


B 




_*'31 


*'32 


''33 _ 


B 



(4) 



10 



Where C^^ to C33 are coefficients having values that all 
typically less then 1. When expanded, expression (4) 
becomes : 



15 



Gc = G~\- C22G -hC2^R-h C2^B 



(5) 



B< = B + c^jB + c^^R + C32G 



Hence, for the image sensor (IS) , the color corrected 
2 0 sampled signals would have the following values for 

each sample: 

Rc^ R + 02R- 0,50-0.258 for channel R; 
Gc = G + 0.4G- 0.15^ -0.255 for channel G; and 
25 Bc = B-h0JB + 0mR-035G for channel B. 

The color correction circuit 3 uses the three 
Low Pass Filters 35,39,40 to produce the R,G,B 
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values. A straightforward application of the Low Pass 
Filters 35^39,40 would have a negative impact on 
image sharpness therefore causing image blur. This 
invention avoids the undesirable introduction of blur 
5 by combining the Low Pass Filters 35,3 9,40 as shown. 

As a result, blur introduced by a single LPF is 
compensated through combining data from several color 
channels. Thus, whilst suppressing the noise, the 
color correction circuitry 3 also preserves the 

10 original sharpness of the images. Uncorrected color 

samples of the Red, Green and Blue color channels are 
respectively applied directly to the adders 34,41,45. 
The low-pass filtered versions of the samples of the 
Red, Green and Blue color channels are multiplied 

15 with the corresponding color correction coefficients 

before being added to the respective uncorrected 
color samples. Thus, the corrected color sampled 
signal (Rq/ ^c) result of a 

combination of the uncorrected color samples and a 

20 noise reduced signal sample that has been low-pass 

filtered and multiplied by selected coefficients. 

Advantageously, the present invention performs a 
method of correcting a digital color sampled signal 

25 comprising a Red Channel, Green Channel and Blue 

channel by filtering uncorrected color sampled 
signals on each of the color channels to provide 
filtered channel sampled signals. The color 

correction circuitry 3 provides for multiplying the 

3 0 filtered channel sampled signals with selected 

coefficients to provide noise reduced signal samples 
of uncorrected color sampled signals. Also, the color 
correction circuitry 3 performs adding each of the 
uncorrected color sampled signals to selected noise 

3 5 reduced signal samples to provide a corrected color 
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sampled signal. Typically, the coefficients for each 
channel, when summed together, are suitably less than 
0.2 and usually the coefficients for each channel, 
when siommed together, are substantially zero. Since 
5 the sum of the coefficients for each channel are less 

than 1, the square of the coefficients are even 
smaller. Since fixed pattern noise is amplified or 
attenuated depending on the value of the square of 
the coefficients, this noise is substantially 
10 reduced. Further, the small size of sum of the 

coefficients for each channel substantially reduces 
blur introduced by the Low Pass Filters 35, 3 9 and 
40. 



15 Although the invention has been described with 

reference to a preferred embodiment, it is to be 
understood that the invention is not restricted to 
the particular embodiment described herein. For 
example, the color interpolation module 4 may be 

20 directly couple to the image sensor 2 and therefore 

provide sampled signals to the color correction 
circuit 3 . 



